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SPECIFICATION 
SYSTEMS AND METHODS FOR PROCESSING X-RAY IMAGES 

BACKGROUND OF THE INVENTION 
Field of the Invention 

[0001] The field of the invention relates to methods and systems for 
processing images, and more particularly, to methods and systems for 
processing x-ray images. 
Background of the Invention 

[0002] Radiation therapy involves medical procedures that selectively expose 

certain areas of a human body, such as cancerous tumors, to doses of radiation. 

The purpose of the radiation therapy is to irradiate the targeted biological tissue 

such that undesirable tissue is destroyed. Radiation has also been used to 

obtain image of tissue for diagnostic or treatment purposes. 

[0003] In a radiation treatment session, the position and movement of a target 

tissue can be monitored by an imaging system, such as a fluoroscopic imaging 

system, while radiation is delivered to the target tissue. This ensures that the 

target tissue is in a desired position while the radiation is being delivered. 

However, often soft tissue targets such as a variety of tumors are not visible in x- 

ray fluoroscopic images. This is due to structures in front or behind the target 

tissue which are also visible in the x-ray images thus increasing the clutter to the 

level that the target tissue cannot be distinguished. 
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[0004] Internal radio-opaque markers have been used to aid physicians in 
identifying a target tissue under fluoroscopic imaging. The radio-opaque markers 
can be injected or implanted at desired sites within a patient, and they shows up 
as high contrast features in fluoroscopic images. By observing the positions of 
the internal radio-opaque markers in fluoroscopic images, a physician can 
determine a position of a target tissue. However, implantation of markers is 
intrusive to the patient, and it may not be practical or feasible in all cases. 
[0005] Accordingly, systems and methods for visualization of internal tissue 
without use of internal markers would be useful. 


SUMMARY OF THE INVENTION 

[0006] In accordance with an embodiment of the present invention, a method 

of processing x-ray images is provided. The method includes collecting a first, 

second, and third x-ray images, determining a composite image based on the 

first and second x-ray images, and adjusting the third x-ray image based on the 

composite image. In one embodiment, the composite image may be determined 

by performing image averaging on the first and second x-ray images. By means 

of non-limiting examples, the averaging may be performed using a boxcar 

averaging technique or based on a weighted average. The adjusting may be 

performed by subtracting the composite image from the third x-ray image. By 

means of non-limiting advantage, the method may be used to enhance a feature 

of an object to thereby allow visualization of the object in the third x-ray image. 
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Such x-ray image processing technique can be used in various procedures in 
. which it is desirable to identify an object under fluoroscopic imaging. For 
examples, the x-ray image processing technique can be used for tracking a 
moving target tissue, or for gating an execution of a procedure based on a 
5 position of a target tissue. Such method does not require the use of markers, 
and can easily be implemented using existing imaging systems. 
[0007] In accordance with another embodiment of the present invention, a 
method of processing x-ray images includes obtaining a first x-ray image, 
obtaining a second x-ray image, and determining a composite image based on at 

10 least a portion of the first and second x-ray images. In one embodiment, the 
composite image may be determined by subtracting the first x-ray image from the 
second x-ray image. By observing the contrast of the composite image, a degree 
of detected motion of an object may be determined. For example, a value 
associated with a contrast of the composite image may be determined, and a 

15 degree of detected motion of an object can be determined based on the value. 
Such x-ray image processing technique can be used in various procedures in 
which it is desirable to detect motion of an object. For example, the x-ray image 
processing technique can be used to detect a motion of a target tissue, and a 
medical procedure may be gated based on the detected motion. Such method 

20 does not require the use of markers, and can easily be implemented using 
existing imaging systems. 
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[0008] Other aspects and features of the invention will be evident from 
reading the following detailed description of the preferred embodiments, which 
are intended to illustrate, not limit, the invention. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

[0009] The drawings illustrate the design and utility of preferred embodiments 
of the present invention, in which similar elements are referred to by common 
reference numerals. In order to better appreciate how advantages and objects of 
the present inventions are obtained, a more particular description of the present 
inventions briefly described above will be rendered by reference to specific 
embodiments thereof, which are illustrated in the accompanying drawings. 
Understanding that these drawings depict only typical embodiments of the 
invention and are not therefore to be considered limiting its scope, the invention 
will be described and explained with additional specificity and detail through the 
use of the accompanying drawings. 

[0010] FIG. 1 illustrates a fluoroscopic imaging system with which 
embodiments of the present invention may be implemented; 
[001 1] FIG. 2 is a flowchart showing a process for targeting an object in 
accordance with an embodiment of the invention; 

[0012] FIG. 3 shows an algorithm for processing images in accordance with 
an embodiment of the invention; 

[001 3] FIG. 4 shows examples of images generated during a treatment or 
diagnostic session performed in accordance with the process of FIG. 2; 
[0014] FIG. 5 is a flowchart showing a process for generating templates that 
may be used in the process of FIG. 2; 
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[001 5] FIG. 6 shows examples of images generated at different stages of the 
template generation process; 

[0016] FIG. 7 is a block diagram showing a system for processing images in 
accordance with an embodiment of the invention; 
5 [0017] FIG. 8 shows a motion signal chart and a gating signal chart; 

[0018] FIG. 9 shows a motion signal chart, a phase chart, and a gating signal 
chart; 

[0019] FIG. 10 is a flowchart showing a process for gating a medical 
procedure in accordance with an embodiment of the invention; 
10 [0020] FIG. 1 1 is a flowchart showing a process for monitoring a patient's 
position in accordance with an embodiment of the invention; and 
[0021] FIG. 12 is a diagram of a computer hardware system with which 
embodiments of the present invention can be implemented. 
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DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

[0022] Various embodiments of the present invention are described 
hereinafter with reference to the figures. It should be noted that the figures are 
not drawn to scale and that elements of similar structures or functions are 
represented by like reference numerals throughout the figures. It should also be 
noted that the figures are only intended to facilitate the description of specific 
embodiments of the invention. They are not intended as an exhaustive 
description of the invention or as a limitation on the scope of the invention. In 
addition, an illustrated embodiment needs not have all the aspects or advantages 
of the invention shown. An aspect or an advantage described in conjunction with 
a particular embodiment of the present invention is not necessarily limited to that 
embodiment and can be practiced in any other embodiments of the present 
invention even if not so illustrated. 

[0023] FIG. 1 illustrates a fluoroscopic system 10 with which embodiments of 
the present invention may be implemented. The system 10 includes a 
fluoroscope 12, a processor 14, and a work station 16 having a display 18 and a 
user interface 20, such as a keyboard and/or a mouse. The processor 14 may 
be an integral component of the work station 16, or alternative, a separate 
component that is connected to the work station 16. The fluoroscope 12 is 
illustrated as a C-arm fluoroscope in which an x-ray source 22 is mounted on a 
structural member or C-arm 24 opposite to an imaging assembly 26, which is 
configured to receive and detect x-ray emitting from the x-ray source 22. The C- 
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arm 24 is capable of moving about a patient for producing two dimensional 

projection images of the patient from different angles. 

[0024] During use of the fluoroscopic system 10, a patient 30 is positioned 

between the x-ray source 22 and the imaging assembly 26. A x-ray beam 32 is 

5 then directed towards a target region 34 within the patient 30, and is attenuated 

as it passes through the patient 30. The imaging assembly 26 receives the 

attenuated x-ray beam 32, and generates electrical signals in response thereto. 

The electrical signals are transmitted to the processor 14, which is configured to 

generate images in the display 18 based on the electrical signals in accordance 

10 with an embodiment of the present invention. During a treatment session, 

another radiation source 28 may be positioned adjacent the fluoroscopic system 
10 for delivering treatment radiation 29 to the target region 34. Similar imaging 
systems or other types of imaging systems may also be used to implement 
embodiments of the present invention. 

15 [0025] FIG. 2 is a block diagram illustrating an embodiment of a process 200 
for tracking a position of the target region 34 of the patient 30 as the target region 
34 is being imaged using the fluoroscopic system 10 of FIG. 1 . 
[0026] To track a position of the target region 34 of the patient 30 undergoing 
a fluoroscopic imaging, a real-time input fluoroscopic image is generated using 

20 the fluoroscopic system 10 (Step 204). The target region 34 may include a 
tissue, such as a lung tissue or a heart tissue, that undergoes periodic 
physiological movements. Alternatively, the target region 34 may also include 

8 

521 1 1 685. 1 /201 8721 -7031 422003 


PATENT 
2018721-7031422003 

tissue that does not undergoes periodic physiological movements, such as a 

bone tissue or prostate. 

[0027] Next, the processor 14 processes the fluoroscopic image to enhance a 
feature, such as a moving feature of an object, in the fluoroscopic image (Step 
206). By enhancing a moving feature in the input fluoroscopic image, contrast of 
an image of a moving object is enhanced while contrast of an image of a 
relatively stationary object is reduced. In the illustrated embodiment, the 
enhancement of the moving feature may be performed based on image 
averaging and image subtraction techniques. 

[0028] In one embodiment, boxcar averaging technique may be used. 
Particularly, to obtain an enhanced input fluoroscopic image EIFI n for the nth 
input fluoroscopic image IFI n , a long term average of the previous input 
fluoroscopic images is calculated and subtracted from the nth input fluoroscopic 
image IFI n , (i.e., EIFI n = IFI n - Avg(IFI x=n . mtox=n . 1 , where m = length of boxcar). 
For example, the sixth input fluoroscopic image IRFI 6 may be enhanced or 
modified by performing image averaging on the previous five input fluoroscopic 
images to obtain a composite image (i.e. an average image), and by subtracting 
the composite image from the sixth input fluoroscopic image RFI 6 . As used in 
this specification, the term "composite image" includes an array of data that may 
be stored in a medium, and therefore, is not limited to a displayed image. 
[0029] In an alternative embodiment, the image averaging may be performed 
based on a weighted average prescribed as a function over time. For example, if 
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later input fluoroscopic images are to be accounted for more in the averaging, 

later input fluoroscopic images may be multiplied by a higher weighted factor 

during the image averaging, and vice versa. FIG. 3 shows a recursive algorithm 

for enhancing a moving feature of an object in an image, in which the current 

input fluoroscopic image is multiplied by a weighted factor (1-a) while the 

previous recursive average of the input fluoroscopic image(s) is multiplied by a 

weighted factor (a). The Z" 1 represents a memory that holds one frame with one 

frame time delay. This results in an exponentially decreasing weighted average 

for the earlier samples. Other types of weighted averaging may also be used. 

[0030] It should be noted that the process of enhancing a feature in the 

fluoroscopic image is not limited to the examples described previously, and that 

other modified versions of the process may also be used. For example, in 

another embodiment, the boxcar averaging may be performed on certain 

previous input fluoroscopic images (e.g., the last three images), instead of on all 

of the previous input fluoroscopic images. In other embodiments, other functions 

or algorithms may be applied to any combination of the previous input 

fluoroscopic images and/or the current input fluoroscopic image before or after 

the image averaging is performed. 

[0031] The processor 14 next registers the enhanced input fluoroscopic image 
with a template (Step 208). In the illustrated embodiment, a sequence of 
templates is provided, and each of the templates contains an image of at least a 
portion of the target region 34 that is created at a certain time-point or a phase of 
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a physiological cycle. The processor 14 selects a template from the sequence of 
templates that best matches an image of the target region 34 in the enhanced 
input fluoroscopic image. The construction of the templates will be described 
later. As used in this specification, the term "phase" refers to a variable that 
represents, measures, or associates with, a degree of completion of a 
physiological cycle. 

[0032] In one embodiment, the input fluoroscopic image is compared with the 
templates, and the template that best matches with an image in the input 
fluoroscopic image is registered or cross correlated with the input fluoroscopic 
image. In this case, the processor 14 performs an image comparison to 
determine which portion of the enhanced input fluoroscopic image best matches 
with each of the template images. Known techniques for performing image 
analysis, such as pattern matching, may be used. For example, if a template 
contains an image formed by 50x50 pixels, the processor 14 may perform a 
spatial analysis to determine a region (having 50x50 pixels) within the enhanced 
input fluoroscopic image that best matches the template image. The processor 
14 then computes values representative degrees of match between the 
templates and an image in the input fluoroscopic image, and selects the template 
associated with the highest value to be registered with the input fluoroscopic 
image. The position of the image within the input fluoroscopic image that best 
matches the registered template may be stored in a computer-readable medium 
for later use. 
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[0033] In one embodiment, each cross correlation between the enhanced 

input image and a template results in a 2D correlation function with a correlation 

peak. In each correlation function, the location of the peak indicates the position 

of the target region 34, and the value of the peak indicates a degree of match 

5 between the input fluoroscopic image and the template. The template that 

provides the highest peak value is then selected as the matching template, and 

the corresponding peak position in the correlation function is used to determine 

the position of the target region 34. 

[0034] Examples of an algorithm that may be used to search for the template 
10 that best matches the input fluoroscopic image will now be described. However, 
it should be understood that the determination of the template that best matches 
the input fluoroscopic image may also be performed using other algorithms or 
techniques. In one embodiment, the input fluoroscopic image is compared with 
all of the templates to determine the matching template. In another embodiment, 
15 instead of comparing the input fluoroscopic image with all of the templates, the 
input fluoroscopic image is compared with only a subset of templates. In this 
case, the subset of templates are selected such that their corresponding phase 
values (or time points of a respiration cycle at which they are generated) are 
centered around, or proximate to, the phase of the template that had the best 
20 match with the last input fluoroscopic image (i.e., from the last tracking cycle). 
Such technique increases the efficiency for registering the input fluoroscopic 
image with the template because an input fluoroscopic image and a template that 
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are collected at the same phase or time-point of a physiological cycle are likely to 
have similar image contrast. In another embodiment, if a match is found 
between the previous input fluoroscopic image and a template, and if the 
templates and the fluoroscopic images are generated at substantially the same 
5 phases or time-points of a physiological cycle, the next template in the sequence 
may be selected to determine if it matches with an image in the current input 
fluoroscopic image. If it is determined that the template does not match the input 
fluoroscopic image (i.e., the degree of match does not exceed a prescribed 
threshold), another template is then selected to determine if it matches with an 

10 image in the input fluoroscopic image. For example, the next template or the 
previous template in the sequence may be selected, until a match is found. 
[0035] Once the input fluoroscopic image is matched with the template, the 
position of the target region 34 in the fluoroscopic image is determined (Step 
210). Particularly, the position of the image in the input fluoroscopic image that 

15 matches with the template is the position of the target region 34. A marker may 
be displayed in the display 18 to indicate the position of the identified target 
region 34 in the input fluoroscopic image. For example, a frame or an outline 
having a similar shape as that of the corresponding registered template may be 
displayed in the input fluoroscopic image. The phase associated with the input 

20 fluoroscopic image can be determined based on the phase of the matched 
template. Alternatively the phase associated with the input fluoroscopic image 
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can be determined by a separate tracking mechanism, such as RPM external 

markers, available at Varian Medical System, Inc., Palo Alto, California. 

[0036] The next real-time input fluoroscopic image is generated and the 

previously described process is repeated until the end of the session is reached 

5 (Step 212). The templates and the input fluoroscopic images may be generated 

at same or different time intervals. For example, the templates may be 

generated at a shorter time interval as compared to that for the input fluoroscopic 

images, thereby allowing more matching variations between different sets of the 

input fluoroscopic images and the templates. 

10 [0037] It should be noted that the steps described previously with reference to 
the process 200 can be carried out in substantially real-time. That is, the input 
fluoroscopic images can be processed to determine a position of the target 
region immediately or shortly after they are generated in step 204. Alternatively, 
the input fluoroscopic images can be generated in a batch, time-stamped, and 

15 stored for subsequent processing. In this case, the enhancing step 206, the 
registering step 208, and the determining step 210 can be performed 
subsequently. 

[0038] FIG. 4 shows examples of images generated at different stages of the 
dynamic targeting process described previously. An example of an input 
20 fluoroscopic image 400 created during a phase of a respiratory cycle, and its 
corresponding motion enhanced fluoroscopic image 402 created using the 
technique described with reference to step 206 are shown. As can be seen in 
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the figure, by subtracting the average image from the current input fluoroscopic 

image, the moving object(s), i.e., the lung tissue 404, is enhanced while the 

contrast of the relatively stationary object(s), i.e., the bone 406, is reduced. FIG. 

4 also shows a rectangular frame 408 displayed in the fluoroscopic image 402 

5 identifying a region in the fluoroscopic image 402 that matches with the template 

410. The template 410 is selected from a group 412 of available templates. The 

group 412 can include all of the generated templates, or alternatively, a subset of 

the generated templates, as discussed previously. 

[0039] The construction of the templates will now be described. Various 
10 methods may be used to generate the templates. FIG. 5 shows a process 500 
for generating the sequence of templates in accordance with an embodiment of 
the present invention. First, the radiation source 22 of the fluoroscopic system 
10 is positioned and aimed towards an area of the body that includes the target 
region 34, and a plurality of reference fluoroscopic images RFI is generated 
15 using the fluoroscopic system 10 (Step 502). The position and orientation of the 
x-ray source 22 relative to the patient 30 may be stored for later use. 
Particularly, the position and orientation of the x-ray source 22 used during the 
template generation session may be used to set up the x-ray source 22 for 
generating the input fluoroscopic images. As a result, the image in the input 
20 fluoroscopic image would be similar to that in the template, thereby allowing 
matching of the template with the input fluoroscopic image. If the target region 
34 includes a moving tissue, the plurality of reference fluoroscopic images is 
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preferably collected over a physiological cycle, such as a respiratory cycle or a 

cardiac cycle, of the moving tissue. In one embodiment, 120 to 200 reference 

fluoroscopic images are collected over a period of 12 to 20 seconds in order to 

capture movements of the target region 34 during a respiratory cycle. The 

collected reference fluoroscopic images are time-stamped and are then stored in 

digital format in a computer readable medium, such as a hard-drive, a CD-Rom, 

a diskette, or a server. 

[0040] Next, the reference fluoroscopic images are associated with phases or 
time-points of a physiological cycle (Step 504). In one embodiment, the 
generated reference fluoroscopic images are time-stamped as they are 
generated in Step 502. A patient position monitoring system, such as that 
available at Varian Medical System, Inc., Palo Alto, California, may be used to 
detect physiological motion of the patient, and generates motion data as the 
reference fluoroscopic images are generated. The reference fluoroscopic 
images are then associated with phases or time-points of a physiological cycle 
based on their corresponding stamped time and the motion data. For example, 
the reference fluoroscopic images can be synchronized with the motion data to a 
common time line. In another embodiment, the reference fluoroscopic images 
may also be registered in phase with three-dimensional computed tomography 
images generated during a planning session (described below). 
[0041] In Step 506, images of the target region 34 are identified in the 
respective reference fluoroscopic images. In one embodiment, the images of the 
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target region 34 may be determined manually by a user, such as a physician or a 
technician. In this case, the user examines each of the selected reference 
fluoroscopic images and identifies the target region 34 in each of the selected 
reference fluoroscopic images. For each identified target region 34 in the 
reference fluoroscopic images, the user may place a marker representative of the 
position of the target region 34 in the corresponding reference fluoroscopic 
image. For example, the user may operate the user interface 20 and place a 
frame around a region of interest (ROI) containing the target region 34 in the 
corresponding reference fluoroscopic image. Alternatively, the user may also 
draw an outline around a ROI having a shape that resembles the target region 34 
in the corresponding reference fluoroscopic image. In this case, the outline may 
represent a boundary of the target region 34 to which treatment may be applied. 
[0042] In another embodiment, the image of the target region 34 in the 
respective reference fluoroscopic images may be determined by projecting a 
three-dimensional (3D) treatment volume onto the respective reference 
fluoroscopic images. In this case, a number of 3D computed tomography (CT) 
images of the treatment volume are obtained such that they cover a period, such 
as a physiological cycle. The 3D CT images may be generated simultaneously 
with the sequence of the reference fluoroscopic images. Alternatively, the 3D CT 
images may be generated separately from the reference fluoroscopic images, in 
which case, the reference fluoroscopic images may subsequently be registered 
in phase with the 3D CT images. Conventional techniques may be employed to 
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register the sequence of the reference fluoroscopic images with the CT images. 

PRM Respiratory Gating System, available at Varian Medical System, Inc., Palo 

Alto, California, may also be used to register the reference fluoroscopic images 

with the CT images. 

[0043] The 3D CT images are then examined to determine the position of the 
target region 34 in the respective images. In one embodiment, the position of the 
target region 34 in each of the respective CT images is projected onto the 
respective two-dimensional (2D) reference fluoroscopic image using known 
transformation techniques. Based on the projected positions of the target region 
34 in the respective reference fluoroscopic images, ROIs containing images of 
the target region 34 can then be defined in the respective reference fluoroscopic 
images. For example, a rectangular frame circumscribing the target region 34 
may be used to define a ROI. Alternatively, an outline having a shape that 
resembles the target region 34 may define a ROI. 

[0044] Next, the reference fluoroscopic images are processed to enhance a 
moving object in the images (Step 508). The enhancement of a moving object 
may be performed using a similar technique described previously with reference 
to the input fluoroscopic images. In the illustrated embodiment, each of the 
reference fluoroscopic images in the sequence is modified based on image 
averaging and image subtraction techniques. Particularly, to obtain an enhanced 
reference fluoroscopic image ERFI n for the nth reference fluoroscopic image RFI n 
in the sequence, a long term average of the previous reference fluoroscopic 
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images is calculated and subtracted from the nth reference fluoroscopic image 

RFI n> (i.e., ERFIn = RFI n - Avg(RFI x=1 tox =n-i). For example, the sixth reference 

fluoroscopic image RFI 6 in the sequence is modified by performing image 

averaging on the previous five fluoroscopic images to obtain an average image, 

and by subtracting the average image from the sixth fluoroscopic image RFI 6 . In 

one embodiment, the image averaging may be performed based on boxcar or 

recursive techniques. In alternative embodiments, the image averaging may be 

performed based on a weighted average prescribed as a function over time, as 

described previously. 

[0045] Next, the images contained within the ROIs in the reference 
fluoroscopic images are stored as a sequence of templates (Step 510). The 
templates may be stored in a computer readable medium, such as a hard-drive, 
a CD-Rom, a diskette, or a server. 

[0046] In the previously described embodiment, the motion enhancement is 
performed after the ROIs are determined in the reference fluoroscopic images. 
However, this needs not be the case. In an alternative embodiment, the order of 
the steps of enhancing a moving object and ROI determination can be different, 
from the process 500. Furthermore, in another embodiment, instead of 
generating reference fluoroscopic images, digitally reconstructed radiographs 
(DRR) are produced from each reference 3D CT image for the direction of 
fluoroscopic image that will be used in treatment. In this case, the target volume 
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is projected in each DRR, and the DRRs are used as the reference fluoroscopic 

images in the same manner as the previous embodiment. 

[0047] It should be noted that the above-described process 500 for generating 

the sequence of templates may be performed in the same session (e.g., a 

treatment session) in which the process 200 is being performed. Alternatively, 

the templates may be generated in another session that is carried out separately 

and prior to a treatment or diagnostic session. 

[0048] FIG. 6 shows examples of images generated at different stages of the 
template generation process 500 described previously. An example of a 
reference fluoroscopic image 600 created during a phase of a respiratory cycle, 
and its corresponding motion enhanced fluoroscopic image 602 created using 
the technique described with reference to step 508 are shown. As can be seen 
in the figure, by subtracting the composite image of previously generated 
reference fluoroscopic images from the current reference fluoroscopic image, the 
moving object(s), i.e., the lung tissue 604, is enhanced while the contrast of the 
stationary object(s), i.e., the bone 606, is minimized. Furthermore, FIG. 6 shows 
a ROI 608 in the fluoroscopic image 602 that has been selected as a template 
610. Note that the input fluoroscopic image 400 described previously with 
reference to FIG. 4 is similar to the reference fluoroscopic image 600 because (1 ) 
the images 400 and 600 are collected from substantially the same angle and 
position relative to the patient 30, and (2) the input fluoroscopic image 400 and 
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the reference fluoroscopic image 600 are collected at substantially the same 

time-point of a physiological cycle. 

[0049] FIG. 7 shows a system 700 for performing the above described 
processes. The system 700 includes a template generation module 702 and an 
image matching module 704, either or both of which may be implemented using 
the processor 14 or a computer system. The template generation module 702 
includes a phase association module 706, which associates the reference 
images 708 with phases or time-points of a physiological cycle. The template 
generation module 702 also includes a projection module 710 that projects a four 
dimensional treatment plan (3D treatment plan overtime) onto the selected 
reference images 708, and a motion enhancement module 712 for enhancing a 
feature in the selected reference images 708. In one embodiment, the motion 
enhancement module 712 enhance a feature in the entire image for each of the 
selected reference images 708. In another embodiment, the motion 
enhancement module 712 enhances a feature in only the projected overlay on 
the selected reference images 708. Also in another embodiment, the motion 
enhancement module 712 is optional, in which case, the system 700 does not 
include the motion enhancement module 712. 

[0050] The image matching module 704 includes a motion enhancement 
module 720 for enhancing a feature in the input images 722 that are generated 
during a treatment or diagnostic session. The image matching module 704 also 
includes a spatial and temporal matching module 724 for matching the input 
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images 722 with the generated templates 714. Particularly, for each of the input 
images 722, the spatial and temporal matching module 724 selects a template 
714 that best matches an image in the input image 722, and generates an output 
726. The output 726 includes the position (X n , Y„) of the sub-image in the input 
image 722 that best matches the template T n , and an index n of the best- 
matching template T n . The index n may be used to determine the time-point or 
phase of a physiological cycle at which the input image 722 is generated. 
[0051] The previously described method allows a user determine a position of 
the target region 34 during a session without the use of a radio-opaque marker, 
and may be implemented using existing imaging systems. The method may be 
used by a physician to perform a wide range of operations or procedures. 
[0052] Dynamic Targeting 

[0053] In one embodiment, the position of the target region 34 obtained using 
the previously described process may be used as an input signal to control and 
aim a radiation treatment beam 29 towards the target region 34. In this case, the 
radiation treatment beam 29 is continuously positioned to follow the target region 
34 based on the positions of the target region 34 identified in the fluoroscopic 
images. For example, the aim point of a treatment radiation beam may be 
controlled by a moving collimator based on data regarding the position of the 
target region 34 received from the processor 14. Alternatively a treatment couch 
supporting a patient can be moved to control a position of the target region 34 at 
which the beam 29 is directed. 
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[0054] Physiological Gating 

[0055] In another embodiment, the above-described method may be used to 
detect a movement of the target region 34, based on which a medical procedure 
may be gated. Several examples of applications towards physiological gating will 
now be described with reference to radiation therapy. However, it should be 
understood by those skilled in the art that similar techniques or methods may be 
used to control other types of treatments or diagnostic procedures. 
[0056] In one embodiment, the radiation source 28 may be gated to be turned 
on or off based on the positions of the target region 34 identified in the input 
fluoroscopic images. In this case, the position of the image within the input 
fluoroscopic image that is registered with the corresponding template may be 
used to determine if the target region 34 has moved beyond a prescribed 
threshold position. If the target region 34 remains within the prescribed threshold 
position, the radiation beam 29 is turned on, and if the target region 34 has 
moved beyond the threshold position, the radiation beam 29 is then deactivated. 
[0057] FIG. 8 shows an example of a motion signal chart 800 and a gating 
signal chart 802 that is aligned with the motion signal chart 800. The motion 
signal chart 800 may be created by using position data of the target region 34 
obtained using the previously described process 200. A treatment interval 804 
may be defined by an upper bound 806 and a lower bound 808, as shown in the 
motion signal chart 800. In the illustrated example, the upper bound 806 has a 
value of 0.8 and the lower bound 808 has a value of -0.8. As shown in the gating 
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signal chart 802, any position of the target region 34 that falls outside the 

prescribed treatment interval 804 results in a "beam off' gating signal 810 that 

stops the application of radiation to the patient 30. Any position of the target 

region 34 that falls within the prescribed treatment interval 804 results in a "beam 

5 on" gating signal 812 that allows radiation to be applied to the patient 30. 

[0058] In another embodiment, the radiation source 28 may be gated to be 

turned on or off based on the phase of a physiological cycle. In this case, the 

position vs. time history of the image within the input fluoroscopic image that is 

registered with the corresponding template may be used to determine a phase of 

10 a physiological cycle. If the target region 34 remains within a prescribed phase 

interval, the radiation beam 29 is turned on, and if the target region 34 has 

moved beyond the prescribed phase interval, the radiation beam 29 is then 

deactivated. 

[0059] FIG. 9 shows an example of a motion signal chart 900, a 
15 corresponding phase chart 902 for the target region 34, and a gating signal chart 
904 that is aligned with the phase chart 902. The motion signal chart 900 may 
be created by using position data of the target region 34 obtained using the 
previously described method (i.e., at step 210). The phase chart 902 may be 
created based on a beginning and an end of a physiological cycle in the motion 
20 signal chart 900. The phase chart 902 shows the phase progression of a 

physiological movement of the target region 34 overtime. A prescribed phase 
interval 906 may be defined by an upper bound 908 and a lower bound 910, 
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which are represented as dotted lines in the phase chart 902. In the illustrated 

example, the upper bound 908 has a value of 185° and the lower bound 910 has 

a value of 25°. According to the illustrated gating signal chart 904, any position 

of the target region 34 corresponding to a phase that falls outside the prescribed 

phase interval 906 results in a "beam off' gating signal 912 that stops the 

application of radiation to the patient 30. Any position of the target region 34 

corresponding to a phase that falls within the prescribed phase interval 906 

results in a "beam on" gating signal 914 that allows radiation to be applied to the 

patient 30. 

[0060] In yet another embodiment, the radiation treatment beam may be 
gated to be turned on or off by associating the templates with treatment data. In 
one embodiment, certain templates may be associated with a "beam on" signal, 
while the rest of the templates are associated with a "beam off' signal. For 
example, templates generated within a prescribed treatment phase interval may 
be associated with a "beam on" signal, while templates generated outside the 
prescribed treatment phase interval may be associated with a "beam off' signal. 
In an alternative embodiment, in addition to the "beam off' and "beam on" 
signals, the treatment data may also include a "beam on duration" signal. In 
other embodiments, the templates may also be associated with treatment data 
that are commonly used in radiation therapy, such as beam shape data and 
radiation dosage data. During a radiation treatment session, real time input 
fluoroscopic images are obtained and are registered with the templates in 
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accordance with the previously described method. When an input fluoroscopic 

image is registered with a template that contains a "beam on" signal, the 

treatment radiation source 28 then directs a treatment radiation beam 29 towards 

the target region 34 for a duration prescribed by the corresponding "beam on 

duration" signal. On the other hand, when an input fluoroscopic image is , 

registered with a template that contains a "beam off' signal, the treatment 

radiation source 28 then holds off the treatment beam 29 and seizes directing 

radiation towards the target region 34. If a template also contains a "beam 

shape" data, when an input fluoroscopic image is registered with such template, 

the processor 14 then directs a signal to a beam-shaping (e.g., a multi-leaf) 

collimator to change the shape of the treatment beam 29 based on the "beam 

shape" data. In one embodiment, to ensure that a correct treatment is being 

delivered to the target region 34, values may be computed to indicate a degree of 

correlation between the previously generated input fluoroscopic images and their 

corresponding registered templates. If the value indicates that there has been a 

high correlation in the temporal and/or spatial matching between the previously 

generated input fluoroscopic images and their corresponding registered 

templates, the registered template for the current input fluoroscopic image is 

likely to be correct, and treatment may be applied in accordance with the 

treatment data prescribed by the corresponding registered template. 

[0061] In yet another embodiment, radiation may be delivered to the patient 

during a desired portion of a physiological cycle. In radiation therapy, it may be 
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desirable to apply the radiation beam 29 towards the target region 34 during a 

portion, such as a quiescent period, of a physiological cycle. For example, 

quiescent periods occur during the respiratory cycle at the ends of expiration and 

inspiration. In this case, the determined position of the target region 34 can be 

5 used to detect quiescent periods of physiological cycles. During the quiescent 

periods, the motion of the target region 34 slows down or may even cease for a 

fraction of a moment, thereby allowing a radiation treatment to be directed to the 

target region 34. 

[0062] It should be noted that in the above described embodiments, the 
10 activation of a radiation beam may be gated in substantially real-time, or 

alternatively, in a predictive fashion. For example, based on a detected position 
of a target region and a degree of match between previous input fluoroscopic 
images and the templates, the processor 14 can predictively activate a radiation 
source (an example of predictive gating) so as to compensate for delay of 
15 activation time inherent in some x-ray systems. Predictive gating has been 
described in U.S. Patent Application Serial No. 09/893,122 referenced herein. 
[0063] FIG. 10 shows a method 1000 for gating a medical treatment based on 
a degree of detected motion of the target region 34 in accordance with an 
embodiment of the present invention. 
20 [0064] To gate a medical treatment on the target region 34 of the patient 30 
undergoing a fluoroscopic imaging, a real-time input fluoroscopic image is 
generated using the fluoroscopic system 10 of FIG. 1 (Step 1004). 
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[0065] Next, a ROI in the input fluoroscopic image is determined (Step 1006). 
In one embodiment, the ROI includes at least a portion of the target region 34, 
which can be a tissue targeted for treatment, or alternatively, any other tissue 
captured in the input fluoroscopic image. The ROI can be determined by a 
5 physician during a treatment or planning session. For example, the ROI may be 
defined by a frame circumscribing a portion of the input fluoroscopic image. 
[0066] Next, a composite image CI is created by subtracting the image in the 
ROI in the previous input fluoroscopic image from the image in the ROI in the 
current input fluoroscopic image (Step 1 008). For example, for the third input 

10 fluoroscopic image IFI 3 generated in a sequence, a corresponding composite 
image Cl 3 is created by subtracting the image in the ROI in the previous input 
fluoroscopic image (i.e., the second fluoroscopic image IFI 2 ) from the third input 
fluoroscopic image IFI 3 (i.e., Cl n = IFI n - IFI n -i). It should be understood that this 
step needs not be performed for the first input fluoroscopic image in the 

15 sequence since there is no previous input fluoroscopic image before the first 
input fluoroscopic image. 

[0067] A value associated with a contrast of the composite image is next 
calculated over the ROI (1010). In one embodiment, the variance of the pixels in 
the composite image, which is associated with a contrast of the composite image 
20 CI, may be calculated over the ROI, and may be used as a measure of the extent 
of motion undergone by the tissue within the ROI (e.g., the target region 34). In 
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other embodiments, different measures of the contrast in the composite image 

may be used. 

[0068] A beam gating signal is determined based on the calculated value 
(1012). Since an image of an object in the ROI having low contrast indicates that 
5 the object has not moved significantly over time, and vice versa, a radiation 

beam may be disabled when the calculated value (associated with the contrast of 
the composite image in the ROI) exceeds a certain threshold, and be enabled 
when the value is below the threshold. In one embodiment, if the calculated 
value m > T . A, then a radiation beam is disabled, and vice versa, where T is a 

10 prescribed threshold value, and A is a normalization factor for compensating for 
changes or daily variations in the operation of the fluoroscopic imaging system 
10. One possible value for A is A = |max m(t) - min m(t)| where max m(t) and 
min m(t) are derived from observing m over a recent physiological cycle, such as 
a respiratory cycle or a cardiac cycle. 

15 [0069] The next real-time input fluoroscopic image is generated and the 
previously described process is repeated until a sufficient radiation has been 
delivered to the target region 34 (Step 1014). 
[0070] Target Object Position Monitoring 

[0071] Besides dynamically targeting a moving object and gating a medical 
20 procedure, methods similar to that described previously may also be used to 
monitor or determine the position of a target object during a session. The target 
object may be a patient or an internal organ. 
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[0072] In one embodiment, a position of the object 30 may be determined 

using a method that is similar to that discussed previously with reference to FIG. 

2. In this case, instead of generating a sequence of templates, one template is 

generated using the process 500 discussed previously. In this case, a portion of 

5 the reference fluoroscopic image containing the target object (i.e., object that is 

not expected to move beyond a certain prescribed threshold during a session) is 

selected as the template. During a treatment or diagnostic session, input 

fluoroscopic images of the target object 30 are analyzed and compared with the 

template to determine the position of the object in the input fluoroscopic images. 

10 For example, the processor 14 may perform image analysis to determine a 
portion in each of the input fluoroscopic images that best matches with the 
template. The position of the matched portion in each of the input fluoroscopic 
images represents the position of the object. By observing the determined 
positions of the object in the input fluoroscopic images, one can determine how 

15 much the target object 30 has moved during a session. With respect to radiation 
therapy, if it is determined that the object 30 has moved beyond a certain 
prescribed threshold, the radiation beam 29 may be deactivated. 
[0073] In certain situations, it may be desirable to determine that there is 
target object movement, and it may not be necessary to determine how much an 

20 object has moved. FIG, 1 1 shows a method 1 100 for target object position 
monitoring (i.e., determining whether there is target object movement) in 
accordance with an embodiment of the present invention. First, the radiation 
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source 22 of the fluoroscopic system 10 and the image detector is positioned and 
aimed towards the target object 30, and a reference fluoroscopic image RFI is 
generated using the fluoroscopic system 10 (Step 1102). 
[0074] Next, a portion of the reference fluoroscopic image is selected as a 
5 ROI (Step 1 104). Particularly, the portion of the reference fluoroscopic image 
should contain an image of a target object, that is expected to be held relatively 
stationary during a treatment or diagnostic session. The position of the ROI in 
the reference fluoroscopic image may be stored in a computer-readable medium 
for later use. 

10 [0075] To perform target object position monitoring during a treatment or 
diagnostic session, a real-time input fluoroscopic image IFI n is generated using 
the fluoroscopic system 10 (Step 1 106). In the illustrated embodiment, the 
reference fluoroscopic image and the input fluoroscopic image are generated in 
the same session with the patient 30 staying in substantially the same position. 

15 Alternatively, the reference fluoroscopic image and the input fluoroscopic image 
may be generated in different sessions. In this case, the x-ray source 22 and 
image detector are set up such that its position and orientation relative to the 
patient 30 are substantially the same as those in which the reference 
fluoroscopic image was generated. 

20 [0076] In Step 1 108, the current input fluoroscopic image IFI n is subtracted 
from the reference fluoroscopic image RFI over the ROI to obtain a composite 
image Cl n (i.e., Cl n = IFI n - RFI). In other words, a portion of the input 
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fluoroscopic image IFI n having the same position as the ROI in the reference 

fluoroscopic image RFI is selected and subtracted from the image in the ROI to 

obtain the composite image Cl n . 

[0077] The composite image Cl n is then analyzed to determine whether there 
5 has been target object movement (1110). If there has been target object 
movement, the pixels in the composite image Cl n should have an increase in 
contrast. The target object 30 may be considered to have moved if the contrast 
increase is above a certain prescribed threshold. With respect to radiation 
therapy, the radiation beam 29 may be deactivated when the contrast increase is 
10 above a prescribed threshold. 

[0078] The next real-time input fluoroscopic image is then generated and the 
previously described process is repeated until the end of the session is reached 
(Step 1112). 

[0079] The above-described target object position monitoring and 
15 determination may be performed in conjunction with the dynamic targeting or 
gating of a medical procedure described previously. Alternatively, other 
techniques for monitoring or determining a target object position, such as those 
described in U.S. Patent Application Serial No. 09/893,122, may also be used. 
The entire disclosure of the U.S. Patent Application Serial No. 09/893,122 is 
20 expressly incorporated by reference herein. 
[0080] Computer System Architecture 
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[0081] FIG. 12 is a block diagram that illustrates an embodiment of a 

computer system 1200 upon which an embodiment of the invention may be 

implemented. Computer system 1200 includes a bus 1202 or other 

communication mechanism for communicating information, and a processor 

5 1204 coupled with the bus 1202 for processing information. The processor 1204 

may be an example of the processor 14 of FIG. 1. The computer system 1200 

also includes a main memory 1206, such as a random access memory (RAM) or 

other dynamic storage device, coupled to the bus 1202 for storing information 

and instructions to be executed by the processor 1204. The main memory 1206 

10 also may be used for storing temporary variables or other intermediate 

information during execution of instructions to be executed by the processor 
1204. The computer system 1200 further includes a read only memory (ROM) 
1208 or other static storage device coupled to the bus 1202 for storing static 
information and instructions for the processor 1204. A data storage device 1210, 

15 such as a magnetic disk or optical disk, is provided and coupled to the bus 1202 
for storing information and instructions. 

[0082] The computer system 1200 may be coupled via the bus 1202 to a 
display 1212, such as a cathode ray tube (CRT), for displaying information to a 
user. An input device 1214, including alphanumeric and other keys, is coupled to 
20 the bus 1202 for communicating information and command selections to 

processor 1204. Another type of user input device is cursor control 1216, such 
as a mouse, a trackball, or cursor direction keys for communicating direction 
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information and command selections to processor 1204 and for controlling cursor 
movement on display 1212. This input device typically has two degrees of 
freedom in two axes, a first axis (e.g., x) and a second axis (e.g., y), that allows 
the device to specify positions in a plane. 
5 [0083] The invention is related to the use of computer system 1200 for 

processing images. According to one embodiment of the invention, such use is 
provided by computer system 1200 in response to processor 1204 executing one 
or more sequences of one or more instructions contained in the main memory 
1206. Such instructions may be read into the main memory 1206 from another 

10 computer-readable medium, such as storage device 1210. Execution of the 
sequences of instructions contained in the main memory 1206 causes the 
processor 1204 to perform the process steps described herein. One or more 
processors in a multi-processing arrangement may also be employed to execute 
the sequences of instructions contained in the main memory 1206. In alternative 

15 embodiments, hard-wired circuitry may be used in place of or in combination with 
software instructions to implement the invention. Thus, embodiments of the 
invention are not limited to any specific combination of hardware circuitry and 
software. 

[0084] The term "computer-readable medium" as used herein refers to any 
20 medium that participates in providing instructions to the processor 1204 for 
execution. Such a medium may take many forms, including but not limited to, 
non-volatile media, volatile media, and transmission media. Non-volatile media 
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includes, for example, optical or magnetic disks, such as the storage device 

1210. Volatile media includes dynamic memory, such as the main memory 1206. 

Transmission media includes coaxial cables, copper wire and fiber optics, 

including the wires that comprise the bus 1202. Transmission media can also 

5 take the form of acoustic or light waves, such as those generated during radio 

wave and infrared data communications. 

[0085] Common forms of computer-readable media include, for example, a 
floppy disk, a flexible disk, hard disk, magnetic tape, or any other magnetic 
medium, a CD-ROM, any other optical medium, punch cards, paper tape, any 

10 other physical medium with patterns of holes, a RAM, a PROM, and EPROM, a 
FLASH-EPROM, any other memory chip or cartridge, a carrier wave as 
described hereinafter, or any other medium from which a computer can read. 
[0086] Various forms of computer-readable media may be involved in carrying 
one or more sequences of one or more instructions to the processor 1204 for 

15 execution. For example, the instructions may initially be carried on a magnetic 
disk of a remote computer. The remote computer can load the instructions into 
its dynamic memory and send the instructions over a telephone line using a 
modem. A modem local to the computer system 1200 can receive the data on 
the telephone line and use an infrared transmitter to convert the data to an 

20 infrared signal. An infrared detector coupled to the bus 1202 can receive the 
data carried in the infrared signal and place the data on the bus 1202. The bus 
1202 carries the data to the main memory 1206, from which the processor 1204 
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retrieves and executes the instructions. The instructions received by the main 

memory 1206 may optionally be stored on the storage device 1210 either before 

or after execution by the processor 1204. 

[0087] The computer system 1200 also includes a communication interface 
5 1218 coupled to the bus 1202. The communication interface 1218 provides a 
two-way data communication coupling to a network link 1220 that is connected to 
a local network 1222. For example, the communication interface 1218 may be 
an integrated services digital network (ISDN) card or a modem to provide a data 
communication connection to a corresponding type of telephone line. As another 

10 example, the communication interface 1218 may be a local area network (LAN) 
card to provide a data communication connection to a compatible LAN. Wireless 
links may also be implemented. In any such implementation, the communication 
interface 1218 sends and receives electrical, electromagnetic or optical signals 
that carry data streams representing various types of information. 

15 [0088] The network link 1 220 typically provides data communication through 
one or more networks to other devices. For example, the network link 1 220 may 
provide a connection through local network 1222 to a host computer 1224 or to 
medical equipment 1226 such as a radiation beam source or a switch operatively 
coupled to a radiation beam source. The data streams transported over the 

20 network link 1220 can comprise electrical, electromagnetic or optical signals. 
The signals through the various networks and the signals on the network link 
1220 and through the communication interface 1218, which carry data to and 
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from the computer system 1200, are exemplary forms of carrier waves 

transporting the information. The computer system 1200 can send messages 

and receive data, including program code, through the network(s), the network 

link 1220, and the communication interface 1218. 

5 [0089] Although the embodiments of the systems and methods have been 

described with reference to fluoroscopic imaging, it should be understood that the 

systems and methods may also be implemented using other types of imaging. 

Depending on the type of imaging used, the previously described methods may 

be modified, and are intended to be within the scope of the present invention. 

10 For example, if the type of imaging technique used is such that it can generate 
images of a target region with sufficient contrast or desired features, then the 
step (i.e., step 206 and 508) of enhancing a moving object in an image may not 
be necessary. Particularly, in other embodiments, if the contrasts or features of 
an image in the templates and the input images are such that they allow 

15 registration between the templates and the input images, then the methods 200 
and 500 may not include step 206 and 508, respectively. 
[0090] Although the methods have been described with reference to radiation 
treatment, it should be understood that the same or similar methods may also be 
used to perform other types of medical procedures. For example, the gating 

20 methods described with reference to FIGS. 8-10 may be used in various 
diagnostic imaging procedures as well as image-guided surgery in which 
movement of surgical instruments are controlled by the position of the target 
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object. In addition, besides real-time and predictive gating described previously, 
the above-described methods may also have applications in retrospective gating. 
In this case, the input fluoroscopic images or the processed input fluoroscopic 
images can be time-stamped and stored for future processing. For example, in 
5 three-dimensional imaging applications such as computed tomography, PET, and 
MRI, physiological data (e.g., position of target region or patient) obtained from 
the processed input fluoroscopic images can be used to retrospectively "gate" a 
reconstruction process. For this purpose, the raw data associated with the 
imaging application is synchronized to a common time base with the 

10 physiological motion data. Segments of the raw data that correspond to 

movement cycle intervals of interest are used to reconstruct the volumetric image 
thus minimizing the distortion and size changes caused by patient motion. 
[0091] Furthermore, the method 200 is not limited to determining a position of 
a portion of a patient or animal body. The method 200 may also be used to 

15 determine a position of a non-animal body or other objects in a medical or non- 
medical environment. 

[0092] Although particular embodiments of the present inventions have been 
shown and described, it will be understood that it is not intended to limit the 
present inventions to the preferred embodiments, and it will be obvious to those 
20 skilled in the art that various changes and modifications may be made without 
departing from the spirit and scope of the present inventions. For example, the 
operations performed by the processor 14 can be performed by any combination 
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of hardware and software within the scope of the invention, and should not be 

limited to particular embodiments comprising a particular definition of 

"processor". The specification and drawings are, accordingly, to be regarded in 

an illustrative rather than restrictive sense. The present inventions are intended 

5 to cover alternatives, modifications, and equivalents, which may be included 

within the spirit and scope of the present inventions as defined by the claims. 
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